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ABSTRACT 

A study examined the underlying factor structure of 
the aptitude tests and work samples being completed by students with 
educational disadvantages (limited reading and mathematics skills) 
who were assessed with the current assessment model in the Akron 
(Ohio) Public Schools. The amount of variance accounted for by the 
factors was also investigated, and clarification was sought to 
determine if any differentiation exists between races and between 
sexes on the assessment measures. The assessment battery studied 
included the following: Apticom Computerized Aptitude and Interest 
Assessment; Valpar Worksample 8, Simulated Assembly; and Valpar 
Worksample 6, Independent Problem Solving. Assessments were completed 
in the students' home schools, with two students tested 
simultaneously, switching positions at the halfway point. The study 
found that the abbreviated vocational evaluation battery is well 
accepted by students and that it provides adequate information to 
initiate vocational planning. The battery meets the needs of the 
school in that it assesses students quickly and is completed in one 
administration. The factor structure obtained suggests that this 
battery does a good job of measuring vital vocational skills. 
However, results could have been skewed by the referral process that 
may have provided more white male students than minority and female 
students and by the fact that the students assessed were in danger of 
dropping out. Further research to determine the impact of race and 
gender is suggested. (Contains 15 references.) (KC) 
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VOCATIONAL ASSESSMENT OF STUDENTS 
WITH DISADVANTAGES: THEIR PECULIAR NEEDS 



BIZBORAH NOLTE, M.S., C.V.E. 

^ Abstract 

If^ This study attempted to examine the rational and components of a shortened assessment battery for 

secondary students with disadvantages. The assessment battery included the Apticon5==^ 
CO Computerized Aptitude and Interest Assessment; Valpar Worksample #8, Simulated Assembly; and 

Q Valpar Worksample #6, Independent Problem Solving. Differences between subtest mean scores 

by sex and by race were reviewed. A factor analysis revealed four factors within the assessment 
battery. These include cognitive skills/general intelligence, perceptual motor skills, clerical skills 
and visual-motor coordination. The factor structure, as expected, holds together. 



Introduction 

Little has been documented in the literature regarding the vocational assessment of students 
with disadvantages since the early 1970's. The term disadvantaged refers to individuals who have 
economic or academic disadvantages and who require special services and assistance in order to 
enable them to succeed in vocational education programs. The term includes individuals who are 
members of economically disadvantaged families, migrants, individuals who have limited English 
proficiency, and individuals who are dropouts from, or who are identified as potential dropouts 
from, secondary school. ( Tederal Register. August 16, 1985 p. 33233). 

Numerous references can easily be obtained regarding vocational assessment of adults and 
students with disabilities. Moed (1960) and Neff (1968) identified five different vocational 
evaluation approaches which are utilized by vocational evaluators in rehabilitation facilities. These 
include: 

• Psychological Testing Approach 

• Job Analysis Approach 

• Worksample Approach 

• Situational Assessment Approach 

• Job Tryout Approach 

Nadolsky (1971) reviewed many of the manpower training programs from the 1960's and 
1970's in his study to develop a model for vocational evaluation of individuals with disadvantages 
that was consistent with practices of vocational evaluators employed in rehabilitation and 
manpower training programs. Nadolsky's general assessment findings can be summarized below: 

• Most psychological tests are not appropriate 

• Relationship between test scores and general occupational requirements should be stressed 

• Emphasize positive aspects of worksample results and compare performance on 
^ worksamples within different occupational areas 

Worksamples are practical, realistic assessments of job skills and provide immediate 



feedback 
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• Situational assessment is most effective if monetary rewards are provided 

• Job tryouts used only after vocational planning from other assessments 

The above constraints are compounded when you also add the component that the 
individual with the disadvantage is a secondary school student. 

The Carl D. Perkins Vocational Education Act Amendments of 1984 required that 
disadvantaged students be provided a vocational assessment at least one year prior to eligibility to 
enter vocational education programs. The most current reauthorization, the 1990 Carl D. Perkins 
Vocational and Applied Technology Education Act, continues to designate a major portion of its 
funds for individuals with disadvantages. Vocational assessment is included in this legislation. 

It is well documented that students with disadvantages do not score well on readily 
available formal tests of intellectual skills (Karp and Sigel, 1965; Society for Psychological Study 
of Social Issues, 1964). Students with disadvantages have more difficulty with reading, 
communication and other skills (Wircenski, 1985). The American Psychological Association Task 
Force on Employment Testing of Minority Groups (1969) found that students with disadvantages 
score substantially lower than others on nearly all aptitudes. 

Freebe;g (1970) acknowledged these assessment difficulties and attempted to create 
alternative assessment measures for students with disadvantages. He acknowledged that the 
assessment must address: 

• Administration format - presentation style or appeal is important to consider 

• Verbal content - maximum difficulty at the 5th grade reading level 

• Pictorial content - to enhance test taking motivation 

• Te^t length - test taking attention span in relatively short 



Smith (1990) also acknowledges physical characteristics of the student with disadvantages: 

• Rapid physical growth - metabolic changes result in increased food and rest consumption 

• Grow of the mind - changes from concrete to abstract logic 

• Search for identity - self concept as a learner develops 

• Development of social skills - learning psychosocial roles 

Beyond all of these accommodations which must be made in attempt to assess this 
population, working with disadvantaged students also requires motivating them to remain in 
school, identifying and removing barriers to student success, and providing basic skills 
remediation, support services, quality vocational training programs, and timely opportunities for 
students to utilize new occupational skills (Dunn, 1987). 

Wircenski (1985) notes that there must be a viable, organized system for identifying the 
skills and abilities of each student with disabilities in the primary and middle school years. He 
goes on to state that the assessment must be on-going into the high school years. The assessment 
should include a complete review of the student's academic skills, learning styles, communication 
skills, aptitudes, vocational interests, family economic conditions, and social adjustment skills. 



The student factor adds unique criteria for the vocational assessment. These characteristics 
are summarized below: 

• Large numbers of students must be assessed as rapidly as possible to limit time away from 
classes. 

• Due to attendance problems assessment must be completed in one sitting. 

• Assessments must provide implications for relevant local training programs and locally 
available employment opportunities. 

• Assessments must be completed at the eighth to ninth grade level in order to implement plans 
that address the scope of services needed at the secondary level. 

• Assessments must be in a form that addresses the limitations of students with disadvantages 
(i.e. limited reading and math skills). 

• Assessment data must be presented to the student and appropriate school and community 
staff and family members that is understandable and relevant to the local employment market. 

• Assessments must address prevocational skills such as on-task behaviors and school 
attendance. 

In combination, these factors document a definite need for development and documentation 
of a workable model for vocational assessment of students that are disadvantaged in secondary 
schools. 

Purpose 

The purpose of this study was to examine the underlying factor structure of the aptitudes 
and worksamples being completed by the students being assessed with the current assessment 
model in Akron Public Schools, Akron, Ohio. Further, the amount of variance accounted by the 
factors was sought to be determined. Clarification was sought to determine if any differentiation 
exists between races and between sexes on the assessment measures. 

Method 

During the last four school years; 1988-1989, 1989-1990, 1990-1991, and 199M992; 
students in the Occupational Work Adjustment (OWA) classes in Akron Public Schools have 
completed a vocational assessment battery. Students in OWA are generally in the ninth grade, 
although a small number of tenth graders may also be in this program. Potential students are 
usually referred by the eighth grade counselors for the OWA program. OWA students meet the 
disadvantaged classification by exhibiting behaviors that cause them to be high risk for dropping 
out of school. Generally, attendance and/or school behavior problems are prominent in the years 
prior to ninth grade. Another reason for OWA placement might be economic need for 
employment. OWA students are in school each morning and are employed in the afternoons. 
Employment is a requirement of the program. There is at least one OWA program in each of the 
eight comprehensive high schools in Akron. 

File records have been maintained for each OWA student completing vocational 
assessment. This study involved only nonreactive file review measures. 



There are 481 complete assessments in this study. 38% (N=183) of the students were 
African-American. 61.3% (N=:295) of the students were Caucasian, and .6% (N=3) were of other 
racial backgrounds. 69% (N=:334) were male students and 31% (N=:147) were female students. 
All students were enrolled in the OWA programs and met the definition of being either 
academically or economically disadvantaged. All students were in the ninth or tenth grade of 
school. 



Instrumentation 



The vocational assessment battery which was completed by the OWA students in Akron 
Public Schools included the Apticom Aptitude Battery; the Apticom Occupational Interest 
Inventory; Valpar worksample #8, Simulated Assembly; and Valpar worksample #6, Independent 
Problem Solving. There are 481 case files of students who completed this battery. There was no 
missing data in any student file. 

The Apticom is a computerized assessment of an individual's vocational aptitude skills and 
vocational interests. According to Bottenbusch (1987), the Apticom system was designed for 
disadvantaged job applicants, high school or special education students, and rehabilitation clients. 
There are eleven subtests in the aptitude assessment ranging in length from 45 seconds to 7 
minutes to complete. Test taking attention span is a factor which must be addressed with this 
population. The aptitude assessment can generally be completed in approximately 45-60 minutes. 
The individual uses a probe to select the correct answer on the electronic board format. A forced 
choice format is utilized. Hardware is utilized for the eye-hand-foot coordination subtest and the 
finger and manual dexterity subtests. The aptitudes are reported in standard scores. 

The Apticom Occupational Interest Inventory is also a forced choice format. It consists of 
162 items using a like-? -dislike format. This section requires 4-5 grade reading level. However, 
items may be read to the individual. It takes approximately 10-30 minutes to complete. A printed 
report is available immediately following administration. Analysis of the interest information is not 
included in this study. 

The Apticom aptitude system is based on the U. S. Department of Labor definitions of ten 
of the eleven aptitudes (except color discrimination), with much of the design and test item format 
closely resembling those of the General Aptitude Test Battery (GATE). 

Aptitude Description 



0 - Intelligence 

V- Verbal 

N - Numerical 
S - Spatial 



P - Form 
Perception 



General learning ability. The ability to "catch on" or understand instructions and 
underlying principles; the ability to reason and make judgments. Closely related to 
doing well in school. 

The ability to understand meaning of words and to use them effectively. The ability to 
comprehend language, to understand relationships between words, and to understand 
meanings of whole sentences and paragraphs. 

The ability to perform arithmetic operations quickly and accurately. 

Ability to think visu?lly of geometric forms and to comprehend the two-dimensional 
representation of three-dimensional objects. The ability to recognize the relationships 
resulting from the movement of objects in space. 

Ability to perceive pertinent detail in objects or 

in pictorial or graphic material. Ability to make visual comparisons and discriminations, 
to see slight differences in shapes and shadings of figures, and widths and lengths of 
Unes. 



Q - Clerical 
Perception 



Ability to perceive pertinent detail in verbal or 

tabular material. Ability to observe differences in copy, to proofread words and 
numbers, and to avoid perceptual errors in arithmetic computation. A measure of speed 
of perception is required in many industrial jobs even when the job does not have verbal 
or numerical contents. 



K- Motor 

Coordination 



Ability to coordinate eyes and hands or fingers 

rapidly and accurately in making precise movements with speed. Ability to make 
movement response accurately and swiftly. 



F - Finger 
Dexterity 



Ability to move fingers and manipulate small 
objects with fingers, rapidly, and accurately. 



M - Mar-ual 
Dexterity 



Ability to move hands easily and skillfully. 

To work with hands in placing and turning motions. 



E - Eye-hand-foot Ability to move the hand and foot coordinately 
Coordination with each other in accordance with visual stimuli. 

The Apticom Aptitude Assessment consists of the following subtests. Following uic 
subtest name is the aptitude(s) it measures and the length of the subtest. Notice that the subtests 
are fairly short, the longest being seven minutes. The number of items in a subtest is also noted. 



Aoticom Subtest 


Aptitude 


Subtest Leneth/Items 


1 


Object idenlificalion 


P 


1 min., 30 items 


2 


Abstract shape matching 


P 


2 min., 30 items 


3 


Clerical matching 


Q 


1 wiri.y 30 items 


4 


Eye-hand-fool coordination 


E 


4j sec. 


5 


Patter visualization 


S &G 


5 min., 30 items 


6 


Compulation 


N 


4 min., 30 items 


7 


Finger dexterity 


F 


2 min. 


8 


Numerical reasoning 


N&G 


7 min., 23 problems 


9 


Manual dexterity 


M 


2 min. 


10 


Word meanings 


V&G 


2 min., 15 sec.; 30 items 


11 


Eye-hand coordination 


E 


45 sec. 



Two Valpar worksamples are also included in the Akron Public School assessment battery. 
These worksamples are basfxl on a trait and factor approach taken from job analysis (Botterbusch, 



Valpar worksample #8, Simulated Assembly, assesses the ability to perform a repetitive 
task requiring finger and manual dexterity. Two plastic washers are assembled onto a pin and 
placed into a rotating wheel. The more parts assembled the higher the score. Incorrectly 
assembled parts are not counted. The worksample is twenty minutes in length. Observations are 
made during the worksample regarding on-task skills, work method, fatigue, bimanual dexterity, 
and the ability to follow directions. 

Valpar worksample #6, Independent Problem Solving, measures ability to perform tasks 
requiring the visual comparison and selection of abstract designs. The individual is presented with 
a set of fifty cards. Each card has five abstract designs of various prin;ary colors. At the top of the 
worksample are three sample cards containing five abstract designs. The task is to compare the 
card to the corresponding sample card and check for abstract design color matches. If the match is 



1987). 



not present a mark must be made with a stylist on the perforated grid. Two scores are obtained. 
Speed of task completion is the quantity score. Errors made during completion Is the quality score. 

The two Valpar worksamples are reported in Method-Time-Measurement (MTM) percent 
scores. MTM is an industrial engineering acronym. According to Todd and Chyatte (1976), MTM 
is a discipline developed in the late 1940's to establish standards of normal performance, improve 
existing methods, and synthesize anticipated pi-oduction. In the Valpar-spective (1976), MTM's 
were discussed in relation to theory and to practice. In theory, MTM is defined as a procedure 
which analyzes any manual operation or method into the basic motions required to perform it and 
assigns to each motion a predetermined time standard which is determined by the nature of the 
motion and the conditions under which it is made. In practice, MTM's provides an objectively- 
derived, entry-level production standard by which an evaluee*s worksample performance can be 
compared. Percent rankings above 100 can be achieved. The MTM norm format accommodates 
performance up to and including 150 percent with 100 percent representing entry level job- 
readiness in terms of what the worksample is designed to measure. 

Procedure 

The assessments were completed in the student's home school. Two students were tested 
simultaneously. Each student verbally completed a short vocational interview with the vocational 
evaluator. Following the interview, one student initiated the Apticom subtests while the other 
student completed Valpar worksample #8 then Valpar worksample #6. The two students then 
switched positions and completed the other half of the assessment battery. The assessments were 
conducted during the morning school time as these students are assigned to work sites in the 
afternoon. The assessment required approximately 2 1/2-3 hours to complete for both students. 
The assessment was completed in entirety, thus not requiring the student to return for additional 
assessment time. Following completion of all students in the individual OWA class, test results 
were shared with the students and individual staffings were held to develop vocational plans. 

Results 

A factor analysis was performed in order to simplify the description of behaviors by 
reducing the categories to a few common traits. A principal component factor analysis was done in 
this case to determine the number of factors/concepts involved in the present battery. Only factors 
with eigenvalues greater than 1 were retained for interpretation. A correlation matrix using unities 
as estimates of communalities served as input. 

Factor loadings are interpreted as correlation coefficients between the variable and the 
factor. The proportion of variance accounted for by a factor measures the overall correlation of the 
variables with the factor. Factor 1 accounted for 29.8% of variance with loadings ranging from 
.35 to .76. Loadings on Factor 2 ranged from -.55 to .49. Factor 2 accounted for 14.5% of 
variance. The third factor had loadings ranging from .05 to .60 and contributed 8.4% of variance. 
The forth factor contributed 8.1% of variance and had loadings of values from .04 to .51. (A 
summary of this analysis is available from the authors.) 

Whereas unrotated factor loadings for each variable are dependent on all the variables in the 
analysis, in rotation the shift is from factors maximizing total variance to factors delineating 
separate groups of highly intercorrelated variables. A varimax rotation was performed and new 
factor loadings emerged. The rotated factor structure suggests that the battery content can be 
described by four constructs. Factor loadings with an absolute value of .44 are defined by the 
authors as meaningful and are presented in Table I. Inspection of significant item loadings 
suggested names for each factor which were descriptive of the skills and abilities involved in each 
measure. Items loadings on the first factor indicate that this is measuring general intelligence. 



Factor 2 encompasses a perceptual motor component. Factor 3 measures clerical skill/attention to 
detail. The construct underlying factor 4 is visual-motor coordination. 

Inspection of Table I shows that the numerical aptitude loads significantly on both factors 1 
and 3 which is reasonable, considering the task involved. The numerical test is made up of a 
calculation component, involving clerical skills and attention to detail, and an applied problem 
component, requiring cognitive skills. The quantity score on Valpar worksample #6 loads 
negatively on both factors 2 and 4. This test encompasses both perceptual motor and visual motor 
skills. On this timed test, the lower the score the better. 

A Hotelling's T 2 multivariate analysis was performed in order to determine if there were 
significant differences between two groups on one or more of thirteen dependent variables, after 
adjustment had been made for possible correlations among the variables. 

The multivariate analysis of sex by race was not significant. However, Hotelling's T^ on 
the two major racial groups differed significantly (p=.000) on at least one of the thirteen dependent 
varii>bles. Table n shows means and standard deviations and also indicates that there were 
potentially seven measures where performance was significantly different. After controlling for 
multiple comparisons, five significant differences remain: general intelligence, verbal aptitude, 
numerical aptitude, the quality score on Valpar worksample #6 and the quantity score on Valpar 
worksample #8. The two groups performed similarly on the other eight measures. 

The Hotelling's T2 multivariate analysis of the effect of sex across the thirteen tests was 
also significant at .000. Inspection of the univariate F tests, given in Table HI suggests that, after 
controlling for multiple comparisons, there were three measures which were statistically 
significant. For spatial and form perception aptitudes, males performed considerable superior to 
females. On clerical perception, female performance was significantly better. One other test, the 
quality score on Valpar worksample #6 approached significance, again with females having the 
highest scores. The two groups performed similarly on all other measures. 

Discussion 

This exploratory study has been an attempt to justify the abbreviated vocational evaluation 
battery. The authors feel that the abbreviated battery is w^ell accepted by the students completing 
vocational assessment. It does provide adequate information to initiate vocational planning. The 
battery meets the needs of the school in that it assesses students quickly and is completed in one 
administration. The factor structure obtained suggests that this battery does a good job of 
measuring vital vocational skills. 

Our sample is, of course, non-random and not representative. Subjects in this study were 
referred to the OWA program from a pool of eighth graders because they were believed to be 
potential drop outs or to have an economic need to work. Eighth grade counselors, from a variety 
of middle schools selected students for a variety of reasons. There are inequities in the percentage 
of male versus female referrals. Selection bias could be contributing to mean differences between 
groups. The discrimination between groups by sex and race may not be due to the test itself. 
Perhaps brighter male students, who have difficulty focusing on traditional subject matter are 
referred while their female counterparts are not. Minority students and females may be screened 
out because they are seen as being served better in more traditional academic environments^ 

The obtained factor structure holds together well. The abbreviated battery appears to 
adequately measure cognitive, perceptual-motor, clerical, and visual-motor skills. Even though 
this factor structure makes sense, it may be unique to the age and ability of the sample, which was 
young, unskilled, and inexperienced. We did not begin this exploratory study with a random or 
representative population. The present sample, with experience, may have yielded a somewhat 



different factor structure. Also a random sample, given the same battery may have broken out the 
factor structure somewhat differently. 

In interpreting the assessment results, clinical judgement should always be utilized in 
addition to the resulting test scores. These scores should be used as a component in the 
determination of a student's vocational goals. Judgement regarding program selection should not 
be made on the basis of any single score. Test scores may be used in association with 
observational reports from other sources and may be weighted by professional judgement along 
with a variety of input from various sources. 

Implications 

There are four major implications for further research. First, the referral criteria needs to be 
investigated. Second, perhaps there is a need to have separate norms, by race and sex, which 
could be developed locally. National norms may discriminate inappropriately, if they are applied to 
samples of students such as those included in the present study. Third, the factor structure should 
be verified with other groups of subjects. Fourth, evaluators need to be aware of the problems in 
applying the usual interpretation of test data to nonrepresentative groups. Evaluators are 
encouraged to use their clinical judgement as well as test scores in making recommendations. 
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